18 . S 096 : Concentration Inequalities , Scalar and Matrix Versions

These are lecture notes not in final form and will be continuously edited and/or corrected (as I am sure it contains many typos). Please let me know if you find any typo/mistake. Also, I am posting short descriptions of these notes (together with the open problems) on my Blog, see [Ban15a]. Concentration and large deviations inequalities are among the most useful tools when understanding the performance of some algorithms. In a nutshell they control the probability of a random variable being very far from its expectation. The simplest such inequality is Markov's inequality: Theorem 4.1 (Markov's Inequality) Let X ≥ 0 be a non-negative random variable with E[X] < ∞. Then, Prob{X > t} ≤ E[X] t. (1) Proof. Let t > 0. Define a random variable Y t as Y t = 0 if X ≤ t t if X > t concluding the proof. 2 Markov's inequality can be used to obtain many more concentration inequalities. Chebyshev's inequality is a simple inequality that control fluctuations from the mean.